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Abstract. 

Homoclinic chaos is usually examined with the hypothesis of hyperbolicity of the critical point. We con- 
sider here, following a (suitably adjusted) classical analytic method, the case of non-hyperbolic points and 
show that, under a Melnikov-typc condition plus an additional assumption, the negatively and positively 
asymptotic sets persist under periodic perturbations, together with their infinitely many intersections on the 
Poincare section. We also examine, by means of essentially the same procedure, the case of (heteroclinic) 
orbits tending to the infinity; this case includes in particular the classical Sitnikov 3-body problem. 
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It is well known that perturbing homoclinic (or hctcroclinic) orbits of dynamical systems may lead to 
the phenomenon of transversal intersections of stable and unstable manifolds of the critical point, and that 
this is one of the routes for the appearance of chaos; it is also known that Melnikov method is a very efficient 
analytical criterion to determine the occurrence of the intersection of stable and unstable manifolds. It is 
impossible to give here even a partial account of the vast literature on this subject, and we will quote here 
only those papers [1-7] which are more strictly related to the present approach (for a more complete list of 
references, see also [8]). 

It can be noticed, however, that almost the totality of the papers dealing with homoclinic chaos and 
Melnikov method considers only the case of hyperbolic critical points (some exceptions will be quoted below). 
We want to show here that a suitable extension of a classical and purely analytical method, used in [5] for 
the hyperbolic case, can also cover - under suitable hypotheses - the nonhyperbolic case. 

For defmiteness, even if more general situations could be considered, we will deal here only with planar 
systems originated from unperturbed Hamiltonians of the form 

H= l -p 2 + V{x) (1) 

We point out that this method can (and will) be applied not only to the case of homoclinic orbits approaching 
nonhyperbolic points corresponding to some "superquadratic" unstable equilibrium points x € R of the 
unperturbed potential V(x), but also to the quite different case of (heteroclinic) orbits tending to ±oo for 
t — ► ±oo respectively, appearing in the case of potentials having an equilibrium point at the infinity. 

For the sake of concreteness, we shall provide the construction and the proof only for the case of 
superquadratic stationary points, but it will appear clear that the argument can be repeated equally well 
for the other one. For what concerns the first case, we refer also to [9], where however a completely different 
approach is used; the second situation includes in particular the classical Sitnikov restricted 3— body problem 
in celestial mechanics, which has been considered in detail by Moser [4], by means of the introduction of a 
singular change of coordinates (see also [10,11]). 

More specifically, we will show that the negatively and positively asymptotic sets of the unstable equi- 
librium point persist even in the nonhyperbolic case and under periodic perturbation. We also obtain that 
the occurrence of an intersection of these sets on the Poincare section (together with the chain of their in- 
finitely many subsequent intersections) can be detected by a Melnikov criterion, plus an additional condition, 
which must be introduced in this case to compensate the lack of the "exponential dichotomy" peculiar of 
the hyperbolic case. 
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1. We consider a Hamiltonian dynamical problem with Hamiltonian of the form (1), plus a smooth pertur- 
bation g depending on one or more real (small) parameters e: 



x = p 

dV (2) 
p = — — h g(e, x, x, t) with g(0,x,x,t) = 

where the potential V(x) of the unperturbed problem is assumed to be analytical and to have a "su- 
perquadratic" unstable equilibrium point at x = xq (say, xq = 0), which may be either a "cubic- like" 
stationary point, or a local maximum for V(x) (in this case, clearly, to < in (3) below): 

„,„, dV(0) d^VfO) „ d'VYO) , nr 

n0) = ^ = -. dx-i ° ; -&y =m ^° some integer > 2 (3) 
We assume that the unperturbed problem possesses a homoclinic orbit doubly asymptotic to xo = 0: 

X = x(t) with lim x(*)=0 (4) 

this happens, e.g., if V(x) also admits at least a stable equilibrium point x\ (e.g., with x\ > x ), and there 
is another point x 2 > X\ such that V(x 2 ) = and dV(x)/dx < for x < x < x\, dV(x)/dx > for 

X\ < X < X2- 

A typical example of this situation is given by a double-well Duffing-type potential, modelling a great 
number of physical situations (this example will be considered explicitly at the end of this paragraph) 

V(x) = -\x* + \x« (5) 

It is immediate to see that the unstable equilibrium point xq = (or better, the point x = 0, p = of the 
first-order problem (2), with a potential satisfying (3)) is not a hyperbolic point, indeed the linearized part 
of (2) (with e = 0) is given by the nilpotent matrix 

1 


(cf. [9,12], where existence and regularity of the invariant manifolds is investigated in a very general setting, 
see also [11]). A first consequence of this fact is that now the "exponential dichotomy" peculiar of the 
hyperbolic situation is no longer available; it is easy to see in fact that the homoclinic solution of the 
unperturbed problem vanishes in our case for \t\ — > oo as 

X(t) ~ \t\-W»-V (6) 

Consider now the effect of the perturbation g(e, x, x, t): here, we must assume first of all that 

5 (e,0,0,t) = (7) 

this ensures that xq = is an equilibrium point also for the perturbed problem. The non-hyperbolicity 
of this point cannot ensure a priori the existence of stable and unstable manifolds; more precisely, the 
non-hyperbolicity has a more radical consequence: we cannot even properly speak of stable and unstable 
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manifolds, indeed, the eigenvalues of the linearized problem are zero, and all the dynamics occurs actually 
on the center manifold (cf. [13]). 

We now want to show that suitable assumptions on the perturbation g may guarantee not only the 
existence of solutions approaching the equilibrium point xq = for t — > — oo and for t — > +00, which we will 
denote respectively by 

x — x D (t) and Xi — Xi(t) 

forming the "out-set" and the "in-set" - and playing the role of negatively and positively asymptotic sets of 
the hyperbolic case - but also the possible presence of transversal intersections on the Poincare sections. 
Let us write the problem (2) in the form (where / = — dV/dx) 

x - f(x) = g(e, x, x, t) (8) 

and let us look for solutions x(t) of the perturbed problem "near" the family of the homoclinic orbits x(t~ ^o) : 
we then put for convenience (see [5]) 

x(t) = x (t - t ) + z(t - t ) (9) 

Inserting into (8) (with the time shift t — to — ► t), we obtain 

z-f'[x(t)}z(t) = G(e,z,z,t + t ) (10) 

where the r.h.s. is given by (we will shortly denote by G = G(e,t,to) this quantity; notice that no approxi- 
mation has been made, nor any term neglected) 

G = G(e, t, t ) = g(e, X (t) + z, X (t) +z,t + t )+ /(*(*) + z) - /(*(*)) - f[x(t)]z (11) 

Consider the homogeneous equation obtained from (10): one solution of is clearly x(i), another independent 
solution ip{t) can be constructed with standard methods (see e.g. [14]) or by direct substitution: these two 
solutions have a different behaviour for t — ► ±00, precisely: 

x(t)~\t\-^-v ^0 

m ~ 1)/(— 2) 

The general solution z(t) (even if in implicit form, through the definition (11) of G) of the complete equation 
(10) can be written in the following integral form, with A, B arbitrary constants and t\ arbitrarily fixed: 

z(t) = A x (t) + Bi/>(t) - X (t) f ip(s)G(e, s, t ) ds + ^(t) f X (s)G(e, s, t ) ds (13) 

Jtx Jt! 

Let us now look for solutions z a (t) (and resp. Zi(t)) of (13) with the property of vanishing for t — > —00 
(resp. t — ► +00); these will provide solutions 

x ,i = X(t) + z a ,i{t) 

of (8) belonging, by definition, to the out-set (resp. the in-set) of x = 0. We have then to require that 

m(B + J t x(s)G(e,s,t ) ds) 
~x{t)U- f\{s)G{e,s,to) ds) 



lim 



lim 



= 

(14) 

= 



where the limits are to be evaluated at — oo when looking for the x Q solutions, and resp. at +00 for the Xi 
solutions. Notice that, in contrast with the hyperbolic case, we have to request here the vanishing of these 
quantities (not only their boundcdncss, as in [5]). 

Now, it is an easy exercise to check, taking also into account the behaviour (12) of the two fundamental 
solutions x(i) and ip(t), that the above conditions (14) are satisfied simultaneously at t = —00 and +00 if 
the following (Melnikov-type condition) 



+ OO 

x(t)G(e,t,to)dt = (15) 

00 



and the additional condition 



lim x(t) f ^(a)G(e,a,to) ds = (16) 



are satisfied. Consider now the linearization of the problem (13) around the solution z(t) = 0, e = 0; in this 
way, condition (15) becomes a computable Mclnikov condition 



*(*) ff(e,x(*),*(t),f + *o) dt = (17) 

-00 

and the same is true for the other condition (16). Once these are satisfied, it is sufficient to apply the 
implicit-function theorem to be granted (see [5]) that there is a regular solution of (10), or equivalently of 
(13), close to the unperturbed solution z = 0, i.e. a solution of (8) close to the homoclinics \(t — to) and 
approaching x = for t — > ±00. 
Assume now also that 

the derivative being evaluated at the point e, t where (17) is satisfied (this is of course "generically" true, see 
[15] for a careful discussion on non-transversal crossings) and that (16) also is satisfied (this may be easily 
fulfilled, see below for this point): there is then, thanks also to (18), a transversal intersection at t = to on 
the Poincare section of the out /in asymptotic sets of x — 0. Using then standard arguments (not based on 
hyperbolicity) , considering the case of perturbations g which are periodic (or almost periodic) functions of 
the time, one immediately deduces that there is an infinite sequence of transversal intersections, leading to 
a situation similar to the usual chain of homoclinic intersections typical of the homoclinic chaos. 

More exactly, we have now to emphasize that, in order to get a more complete description of the 
dynamical behaviour in this case, we cannot directly refer to the classical Birkhoff-Smale theorem on the 
equivalence with the Smale horseshoe dynamics, because the proofs of this equivalence are intrinsically based 
on hyperbolicity properties [2-4,6,7]. Some short comment about this point will be given at the end of this 
paper. Here, we can qualitatively say that the presence of infinitely many intersections indicates at least the 
presence of a quite complicated geometry in the behaviour of the dynamical flow, i.e. a sort of "nonhyperbolic 
homoclinic chaos" (cf. [9-13]). 

We now give some conditions to ensure that (16) is satisfied. We assume here for instance that the 
perturbation g can be written in the form 

g(e,x,x,t) = eix" 1 sinwt + e 2 x™ 2 (19) 
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(notice, incidentally, that if the term with e 2 describes a damping and n 2 is even, then instead of x n2 one 
must more correctly write |i|a;™ 2_1 , with e 2 > 0, of course), it is easy to check, recalling also (12), that if 

2(v - 2) 

m > v - 2 and n 2 > '- (20) 

then condition (16) is satisfied. 

Considering e.g. the following explicit example, with V given by (5), 

x = 2x 3 - 3x 5 + e 1 x 3 smut + e 2 x 3 (21) 

there are two homoclinic orbits of the unperturbed Hamiltonian, given by 

±1 



x(t) = 



3expH 

32w(l + w) V ; 



conditions (20) are satisfied (here v = 4) and the Melnikov condition (17) can be explicitly evaluated, giving 
transversal homoclinic intersections if 

ei 
£2 

A numerical integration of this problem, performed along the same lines as in [6, Chapt.2] (see [8] for details), 
shows that, choosing e.g. uj = 1, e 2 = .05, transversal intersections appear when ei>,.006, in quite good 
agreement with the value ei = .00637 given by (22). 

Summarizing, we can state the above results in following form: 

Theorem. Consider a planar dynamical system of the form (2) where the potential V{x) admits a su- 
perquadratic unstable equilibrium in some point, say x = 0, producing then a nonhyperbolic unstable 
equilibrium point for the unperturbed system (given by e = 0). Assume that the unperturbed system ad- 
mits a homoclinic orbit x(i) doubly asymptotic to xq. Assume for simplicity the perturbation g of the 
form (19) with the conditions (20) satisfied. If in addition Melnikov conditions (17) and (18) are satisfied, 
then the perturbed problem admits a negatively and positively asymptotic sets of the unstable equilibrium 
point, which admit an infinite sequence of homoclinic intersections on the Poincare section, giving rise to a 
(nonhyperbolic homoclinic) chaotic dynamical flow. 

2. Instead of discussing some of the possible extensions of the above results to more general dynamical 
systems or to problems in greater dimension, we prefer here to examine a quite different situation which in 
fact can be discussed with a similar procedure. Let us consider a problem with Hamiltonian (1), where now 
the potential is such that 

V(x) < , VieR with lim V(x) = (23) 



and there is only a stationary point xq (a minimum) for V(x)\ 

dV^o) _ Q 
Ax 
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(23') 



There is then a (heteroclinic) orbit approaching — oo and +00: 



X = X(t) with lim X (t) = Too (24) 

A well known example is given by the classical Sitnikov restricted 3— body problem in celestial mechanics, 
which - at the limit of zero eccentricity - is described by the potential [4] 

V(x) = - r L= (25) 

More in general, we can consider a potential such that 

V(x) <~ — ^ for \x\ — ► 00 with fi > 0, real (26) 

Clearly, these problems are, per se, nonhyperbolic, the equilibrium point being at the infinity; actually, 
Moser [4] examined the above problem (25), in the presence of the periodic perturbation produced by 
nonzero eccentricity, using the McGehee [16] singular coordinate transformation defined by x = 2/y 2 and 
di = (4/y 3 ) ds, which in fact transforms the problem into a hyperbolic one near the point y = 0, and he was 
able to prove the existence of Smale horseshoe dynamics (see also [10,11]). 

Our above procedure can be equally well used in this new case, i.e. for problems with potential of the 
form (23,27): indeed, writing the problem as in (8), we can again introduce the variation equation (10-11). 
In this case, one finds that two independent solutions of the homogeneous part of (10) have the following 
behaviour, for \t\ — > 00 

x(t) ~ |t|-"/( 2 +") -» 

(27) 

lP(t) ~ | t |(2+2*0/(2+M) oo 

This allows us to repeat exactly the same arguments as in part 1. In particular, assuming e.g. a non- 
dissipative periodic perturbation of the form, for large 

g(e, x, x, t) ~ ex~ n sinujt (28) 

it is easy to show that the existence of out /in sets is granted if 

n>2 + n (29) 

and to show also that the Melnikov function (17) possesses infinitely many transversal zeroes. Notice that 
the Sitnikov 3— body problem above mentioned falls into this situation, indeed at the first order in the 
eccentricity e - the perturbation is given by [4] 

—3a; 

■ cost 



A(x 2 + 



if" 



As already remarked in part 1, the standard Birkhoff-Smale theorem cannot be directly used for this 
nonhyperbolic case. Here, however, at least for the case of Sitnikov problem, we can refer to the arguments 
used in [10] (see especially the Appendix of [10], and also [13]), to obtain an equivalence to a "nonhyperbolic 
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horseshoe" , where contracting and expanding actions are not exponential but "polynomial" in time. Alterna- 
tively, in the general case, one may possibly resort to the method of "blowing-up" , devised to investigate the 
properties of nonhyperbolic singularities by means of suitable changes of coordinates [17,18], but presumably 
a full and general treatment of nonhyperbolic horseshoes is still open. 
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